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ABSTRACT 

The  fundamental  problem  of  target  tracking  is  to  estimate  the  state  of  one  or  more  objects  that  persist  over 
time ,  based  on  noisy  measurements  contained  in  a  vast  quantity  of  mostly  spurious  measurement  data. 
Target  tracking  is  closely  related  to  a  number  of  basic  problems  in  statistical  modelling  and  information 
extraction  from  noisy  data.  Multi-stage  processing  provides  a  wealth  of  processing  options  that  can  be 
exploited  to  achieve  robust  and  high-performance  surveillance.  This  manuscript  describes  a  number  of 
multi-stage  tracking  architectures  that  the  author  has  recently  studied.  Additionally,  we  study  the  target 
cardinality  problem. 


1  INTRODUCTION 

Tracking  combines  estimation  theory  and  detection  theory,  with  the  further  complication  of  measurement 
origin  uncertainty.  Table  1  identifies  a  number  of  fundamental  problems  in  information  extraction  from 
noisy  data. 


Table  1 .1 :  A  taxonomy  of  information  extraction  problems  in  target  tracking.  The  problems 
noted  in  boldface  are  addressed  in  this  lecture. 


Number  of 
targets 

Number  of 
targets 

False 

contacts 

Target 

motion 

Solution  methodology 

single 

known 

no 

parametric 

non-Bayesian  estimation  (least  squares) 

single 

known 

no 

stochastic 

Bayesian  estimation  (nonlinear  filtering) 

single 

known 

yes 

parametric 

non-Bayesian  estimation  (maximum- 
likelihood  tracking) 

single 

known 

yes 

stochastic 

Bayesian  estimation  (single-target  track 
maintenance) 

multiple 

known 

yes 

stochastic 

Bayesian  estimation  (multi-target  track 
maintenance) 

multiple 

unknown 

yes 

N/A 

Bayesian  estimation  (target  cardinality) 

multiple 

unknown 

yes 

stochastic 

Hypothesis  testing  &  Bayesian 
estimation  (coupled  track  management 
&  track  maintenance) 

As  we  see  from  table  1.1,  many  of  the  solution  methodologies  relevant  to  the  surveillance  problems  of 
interest  rely  on  estimation  and  detection  theory.  Here,  we  will  focus  on  our  recent  work  in  the  last  two 
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problem  domains  listed  in  table  1.1.  First,  in  section  2,  we  study  the  target  cardinality  problem.  Next,  in 
section  3,  we  overview  a  number  of  multi-stage  and  multi- hypothesis  data  fusion  architectures  that  we 
have  developed  in  recent  years,  and  provide  details  and  examples  for  some  of  these. 


2  THE  TARGET  CARDINALITY  PROBLEM 

Many  of  the  problems  in  table  1.1  involve  the  application  of  estimation  theory  for  determining  where  a 
target  or  multiple  targets  are  to  be  found,  assuming  that  the  number  of  targets  is  known.  Now  we  wish  to 
determine  the  number  of  targets,  i.e.  target  cardinality,  disregarding  the  question  of  where  targets  are 
located.  The  solution  to  this  problem  constitutes  a  further  application  of  estimation  theory. 


2.1  The  static  problem 

Assume  that  the  surveillance  region  contains  NT  target,  where  Nr  ~  Poisson{fT) .  That  is, 

2  nT 

p(NT -nT)-—!—  exp(-Tr),  nT  >0  .  (2.1) 

nT\ 

Further,  assume  that  targets  are  detected  with  probability  p.  Invoking  the  splitting  property  of  the  Poisson 
process  [1],  we  have  that  the  probability  distribution  for  the  number  of  target  contacts  N c  is  given  by 
Nc  ~  Poisson(p/LT ) : 


P{Nc  =nc)  = 


(p^T  T1' 


nT\ 


exi 


P (-P*r),  nc  >°- 


(2.2) 


A  question  of  interest  is  the  following:  what  is  the  probability  distribution  on  Nr,  given  that  nc  contacts 
are  observed?  The  distribution  of  interest  can  be  obtained  by  use  of  Bayes’  rule  and  the  probability 
distribution  of  Nc  given  NT ,  which  is  given  by  the  binomial  distribution. 

i  \t  \_p{Nc=nc\NT  =  nT)p{NT  =nT) 

p[NT  —nT  \Nc  ~nc)~  /.r  \ 

P\N c  ~nc) 


nT  >  nc . 


p(Nc  -  nc  |  Nt  =nT)= 


\ncj 


(1  -p)' 


nc  >  0  and  nc  <  nT . 


(2.3) 

(2.4) 


Manipulation  of  (2. 1-2.4)  yields  the  following. 

p(nt =„r \NC =„c)=(NzN)"'^  exp(-(i-pyr) 


nT  >  nc . 


/  \.  5  "T  —  "C '  (2-5) 

\nT  -ncf. 

That  is,  having  observed  nc  contacts,  the  number  of  targets  equals  or  exceeds  nc  with  a  probability  that 
is  prescribed  by  (2.5);  thus  NT  =nc  +  r ,  where  r  ~  Poisson((  1  -  /;)/l7  ).  This  is  a  nice  result,  in  that  the 
number  of  unobserved  targets  obeys  the  same  probability  distribution  as  we  had  a  priori,  albeit  with  a 
suitably  reduced  mean. 

Remark  2.1  For  some  surveillance  problems,  notably  in  the  mine-warfare  community,  the  Poisson  prior 
assumption  is  not  always  invoked.  The  assumption  of  a  uniform  prior  leads  to  a  negative  binomial 
distribution  [2],  Modifications  to  the  uniform  prior  have  been  considered  to  reason  over  multiple 
surveillance  regions  [3].  Another  philosophy  for  choosing  the  prior  distribution  leads  to  the  Katz 
distribution  [4]. 

Remark  2.2  One  criticism  of  the  Poisson  distribution  is  that  does  not  address  the  interdependencies  that 
may  exist  with  respect  to  target  presence.  In  some  applications,  as  in  mine  warfare,  it  is  found  empirically 
that  the  conditional  distribution  on  undetected  targets  depends  on  the  number  of  detected  targets:  targets 
exist  in  clusters.  A  second  criticism  is  that,  as  a  one-parameter  distribution,  its  variance  cannot  be  set 
independently  of  the  mean  (they  are  the  same),  and  thus  does  not  accurately  reflect  uncertainty. 
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Remark  2.3  An  important  characteristic  of  the  solution  given  by  (2.5)  is  that  it  is  consistent  with  respect 
to  partitioning  of  the  surveillance  region.  That  is,  consider  two  non-overlapping  regions  with  priors  Arx 
and  AT2 ,  in  which  we  observe  ncx  and  ncx  targets,  respectively.  Assuming  the  same  detection 
probability  p  applies  to  both  regions,  we  have  Nn  =  ncl  +  r,  and  NT2  -nc 2  +  r2 ,  where 
rx  ~  Poisson((\  -  p)XTX )  and  r2  ~  Poisson((\  -  p)XT1).  Summing  these  results,  we  have 
Nn  +  NT2  =  ncl  +  nc  +  r  ,  where  r  —  rx  +  r2  ~  Poisson((  1  -  pfA T1  +  An ))  using  the  merging  property  of 

the  Poisson  process  [1].  This  is  the  same  result  that  one  obtains  by  considering  a  single  surveillance 
region  given  by  the  union  of  the  two  surveillance  regions. 

We  now  consider  the  multi-sensor  (or  multi-scan)  problem,  in  which  a  region  with  a  fixed  number  of 
targets  Nr  is  observed  S  times,  resulting  in  a  sequence  of  detection  cardinalities  that  we  denote  by 
nc  =(nc\,nc2’--’ncs) ■  We  are  interested  in  the  same  question  as  before:  what  is  the  probability 
distribution  on  NT ,  given  that  the  sequence  nc  is  observed? 

The  measurements  in  the  sequence  nc  are  conditionally  independent,  given  the  number  of  targets  Nr. 
Accordingly,  (2.4)  can  easily  be  replaced  by  (2.7).  Then,  applying  Bayes’  rule,  the  probability  distribution 
on  Nt  given  that  the  sequence  nc  is  observed  is  given  by  (2.8),  which  relies  on  (2.1)  and  (2.7). 


p(Nc  =  nc  |  Nt  =/ir)  =  f] 


=i  \naj 


0  -  pY 


lCi 


>  QVi  and  max{»0  }<, 


p(NT=nT\Nc=nc)=  =  =  _  „ 

Y,p(NC  =nc\  NT  =  j)p(NT  =  j ) 


r  >ma x{na: 


(2.7) 

(2.8) 


y=max(na} 


Unfortunately,  the  posterior  probability  distribution  given  by  expression  (5.8)  does  not  lend  itself  to  a 
simple  interpretation  as  we  had  with  (2.5). 

Remark  2.4  A  recursive  implementation  of  (2.8)  is  possible.  Let  «c  ~  {nci’nc2’—’nc(s-\))’  so  ^at 

nc  -  {nc,ncs ).  Then,  using  the  conditional  independence  of  measurements  in  the  sequence  nc  given  the 
number  of  targets,  (2.9)  results. 

p(NT  =nT\Nc  =nc)  =  - 


■>{^cs  ~  ncs  I  Nc  —  nc ,  Nt  —  nT  ^)p(nt  —  nT  \  N c  —  nc ) 


-  / 

's  ’feC 

pi^cs  ~ 


lcs 


I  Nc  —  nc ,  Nt  —  j)p[N j  —  j  |  Nc  —  nc ) 


/=max<  nrc ,  max  n„ 

J  1  C”i6{l„..,S-l}  c 


ncs  1  Nt  —  nr  )p(Nt  ~  nT  I  Nc  —  nc ) 


(2.9) 


Z  p(Ncs  =  ncs  I  nt  =  j)p{nt  =  j  I  Nc  =  nc ) 

y=max{»c,itemaxi}«d} 

Having  determined  the  posterior  probability  distribution  as  prescribed  by  (2.5),  (2.8),  or  (2.9),  we  are  now 
in  the  position  to  estimate  the  target  cardinality.  We  can  do  so  with  a  number  of  Bayesian  estimators, 
including  MMSE,  MMAE,  and  MAP.  The  MAP  estimator  is  the  simplest,  as  maximization  of  the 
posterior  distribution  over  nT  does  not  require  computation  of  the  denominator  in  (2.5),  (2.8),  or  (2.9). 


5.2  The  static  problem  with  false  alarms 

We  consider  now  a  generalization  of  the  problem  considered  in  section  5.1.  Again,  we  have  an  unknown 
number  of  targets  with  prior  distribution  given  by  NT  ~  Poisson[AT) .  At  each  scan,  in  addition  to  target 
detections  which  occur  with  probability  p  for  each  target,  we  observe  a  number  of  false  returns  with  prior 
distribution  N FA  ~  Poisson(AFA ) .  As  before,  we  observe  a  sequence  of  S  detection  cardinalities  given  by 
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nc  =(nc\,nc2’“-’ncs) ■  What  is  the  probability  distribution  on  NT,  given  that  the  sequence  nc  is 
observed? 

It  turns  out  that  this  problem  can  be  answered  with  minor  modifications  to  our  previous  results,  by 
conditioning  on  the  number  of  returns  that  are  target-induced  in  each  scan  to  obtain  (2.10). 


S  fir 


p(Nc  —  nc  | Nt  -nT)—  ^ 


i= i  j= i 


2' 


nc~J 


exi 


P  {-*fa) 


inCi  ~j) 


1  J 


'(1  ~P) 


nT~J 


,  na  >  0 V/ ,  nT  >  0  . 


V  v  u  j  /  \  j  y  j 

,J*r  u,  \_p(Nc=nc\NT=nT)p{NT=nT)  ^ 
p[N T  —  nT  |  N c  —  nc  )—  oo  ,  nT  >  0  . 

X  p(Nc  =nc\NT=  j)p(NT  =  j) 
j= 0 


(2.10) 

(2.11) 


In  the  single-scan  case,  applying  both  the  merging  and  splitting  properties  of  the  Poisson  process,  the 
denominator  in  (2.1 1)  can  be  expressed  more  simply  according  to  (2.12). 


p{Nc  =  n  c )  =  ^ FA  +  PfT  -  exp(-  Afa  - pAT ),  nc>  0. 


nc\ 


(2.12) 


As  before,  once  the  posterior  probability  distribution  is  defined,  Bayesian  estimators  can  be  readily 
computed. 

Remark  2.5  In  the  mine -warfare  community,  the  general  problem  with  false  alarms  is  not  usually 
considered  [2-4],  The  reason  for  this  is  that  detection  processing  includes  close  inspection  of  suspected 
mines,  so  that  false  alarm  rates  are  negligible.  Nonetheless,  the  generalization  is  of  interest  in  other 
surveillance  settings. 

Remark  2.6  As  before,  a  recursive  computation  of  the  posterior  probability  distribution  for  NT  is 
possible,  according  to  (2.13). 

p(NT  =  nT  \NC  =nc)=  PjNcs  =  Hcs  1  =  "r  I  Nc  -  »c )  _  (2.13) 

,  p{Ncs  ~ncs  I  Nt  —  j)p{^T  ~  J  I  Nc  =  nc  | 
j= 0 

5.3  The  dynamic  problem 

We  consider  now  a  generalization  of  the  target  cardinality  problem,  whereby  the  number  of 
targets  is  time -varying,  as  specified  by  birth  and  death  processes.  In  particular,  assume  that,  at 
each  scan,  the  number  of  target  births  is  Poisson  distributed  with  parameter  AB,  and  that  the 


number  of  target  deaths  is  Poisson  distributed  with  parameter  q  ■  AT,  i.e.  at  each  scan  each  existing  targets 
dies  with  probability  0<?<1.  Once  more,  we  observe  a  sequence  of  S  detection  cardinalities  given  by 
nc  =  (nci,nc2,...,ncs),  and  we  wish  to  determine  the  probability  distribution  on  Nr,  given  that  the 
sequence  nc  is  observed. 

In  the  absence  of  cardinality  measurements,  the  steady-state  target  distribution  is  uniquely 
detennined  by  the  parameters  AB  and  q  .  In  particular,  equating  the  birth  and  death  rates,  equation 

(2. 14)  follows.  Thus,  we  assume  that  the  prior  target  cardinality  is  given  by  NT  ~  Poisson(/iT ) ,  with  Ar 
consistent  with  (2. 14). 

(2.14) 

<7 


Remark  2.7  More  generally,  given  a  target  distribution  with  parameter  A0 ,  after  k  scans  we  have 

(2. 15).  It  is  easy  to  show  from  (2.15)  that,  for  k  — >  oo  ,  we  have  x. (/v )  — >  Ar .  It  is  interesting  to  note  that 
arbitrarily  long  back-prediction  consistent  with  the  dynamics  (2.15)  is  only  possible  for  target  densities 
that  exceed  AT  .  The  back-prediction  equation  and  the  associated  back-prediction  limit  are  given  by  (2.16- 
2.17).  Note  that  the  limit  in  (2.17)  only  applies  to  target  cardinalities  lower  that  the  steady-state  solution; 
when  these  are  larger,  arbitrarily  large  back-predictions  are  possible. 
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k- 1 


*  (k)  =  'to  (i  -  qf  +  X  ab  (!  -  q)‘ 

i=0 


Xq  —  - 


/=0 


(1-*)A 


k- 1 


A0  >0^A(k)>'^AB(l-q)i  <=>A(k)>Aa  1  ^  q'  ok< 


log 


l_qM/c) 
A 


An 


B  '  for  A(k)<  — -  . 

q 


(2.15) 


(2.16) 


(2.17) 


i=o  q  log(!  -  q) 

We  now  develop  a  recursive  expression  for  the  posterior  distribution  on  target  cardinality,  given  by  (2.18- 
2.19).  Compared  with  (2.9),  it  requires  the  diffusion  or  prediction  equation  (2.18),  which  involves 
conditioning  on  the  number  of  targets  that  survive  from  the  previous  scan.  Equation  (2.19)  involves 
conditioning  on  the  previous  target  cardinality;  note  that,  due  to  the  death  process,  only  the  last 
measurement  is  relevant  for  providing  a  lower  bound  on  target  cardinality.  Equation  (2.19)  requires  (2.4). 


p(n„ = j  I  = >■)= ±r‘  (1-?)* 


k= 0 


(2.18) 


p{NTs  —  nrs  I  Nc  —nc)~ 


co  .  . 

p(.NCs  ~  ncs  I  NTs  -  nrs  )5^,p{Nts  ~  nTs  I  NT(s- i)  =  i)/^(/Vr(iS_i)  =  i  \  Nc  =ncj 


i= 0 


°°  00  ,  ,  /  \ 

y  pi^cs  =  ncs  I  =  7)y  Px^ts  ~  j  I  ^r(s-i)  =  =  *  I  ^ c  ~nc) 


J=ncs 


i= 0 


(2.19) 

Finally,  we  consider  a  further  generalization  to  include  both  cardinality  dynamics  and  false  alarms. 
Equations  (2.18-2.19)  are  directly  applicable;  in  (2.19),  in  place  of  (2.4),  we  use  (2.20)  -  the  single-scan 
version  of  equation  (2.10). 


p{N  c  ~  nc  I  —nr)—  y 

j= i 


XpA  1  ex 


P(  ^  FA  ) 


("a  -  J')- 


\ 


W 


(i-^r 


■t-j 


,  nc  >  0,  nT>  0  .  (2.20) 


3  RECENT  ADVANCES  IN  MULTI-STAGE  TRACKING 

Among  the  known  approaches  to  multi-target  tracking,  multi-hypothesis  tracking  (MF1T)  appears  to 
provide  the  best  performance  results,  albeit  at  the  cost  of  increased  computational  expense  and  with  some 
(generally  minor)  time  latency  between  the  input  and  output.  Elowever,  single-stage  centralized  MEIT 
processing  does  not  suffice  in  a  number  of  surveillance  settings.  Table  3.1  lists  a  number  of  multi-stage 
fusion  architectures  that  we  have  studied,  with  a  brief  mention  of  their  applicability.  Further  details  on  our 
MFIT  algorithm  and  on  many  of  the  multi-stage  architectures  listed  in  table  3.1  may  be  found  in  [5]  and 
references  therein,  as  well  as  in  more  recent  papers  [6-7]. 
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Table  3.1:  Multi-stage  data  fusion  architectures. 


Architecture 

Applicability 

track-while-fuse 

Baseline,  centralized  MFIT  approach  [5] 

track-before-fuse 

Fligh  detection-threshold  settings  with  pronounced 
target-fading  effects,  as  in  undersea  surveillance  [5] 

fuse-before-track 

Large  sensor  networks  with  synchronized  sensors; 
simplest  first-stage  static-fusion  approach  is  contact 
sifting  [5] 

track-  extract-track 

Dim  targets  in  clutter  [5] 

track-break-fuse 

Dense  target  scenarios  [5] 

modified  fuse-before-track 

Large  sensor  networks  with  synchronized  sensors; 
approaches  include  ML-MFIT,  ML-MF1T2,  and 
MFIT2  [6] 

track-segment-fuse 

Dense  target  scenarios  with  lengthy  group  motion 
and  low-revisit  rate  feature-rich  sensors  (see  below; 
closely  related  to  the  approach  described  in  [8]) 

modified  track-before-fuse 

Multi-sensor  settings  where  feature-rich  sensors 
have  low  revisit  rates  or  intermittent  coverage, 
requiring  customized  track-management  logic  [7] 

track-  adapt-track 

Low-resolution  sensor  settings  (see  below) 

low-q  track  -  high-q  track 

Dim  targets  in  clutter  (see  below) 

track  merging 

Post-processing  clean-up  procedure  that  relaxes 
single-detection  assumption  (see  below) 

The  motivation  at  the  heart  of  all  the  multi-stage  approaches  outlined  in  table  3.1  is  to  identify  multi-stage 
approaches  that  match  or  exceed  the  performance  and  robustness  characteristics  of  much  more  complex 
single-stage  tracking  solutions.  Further,  the  MF1T  algorithmic  module  at  the  heart  of  these  architectures  is 
highly  modular,  thus  providing  simplicity  and  flexibility  of  use  in  addressing  problems  in  a  wide  range  of 
surveillance  settings.  We  discuss  some  recent  developments  below. 

3.1  Hypothesis  management  in  dense-target  scenarios  ( track-segment-fuse ) 

As  a  computationally-efficient  variation  on  the  track-break-track  architecture  [5],  we  may  consider  the 
following  scheme.  A  first  MF1T  stage  (with  n-scan=0)  identifies  high-quality  contact  data.  Subsequently, 
data  associations  are  broken  only  when  group  compositions  change.  That  is,  in  the  second  pass,  all  tracks 
are  grouped  by  normalized  state  proximity  and,  when  a  new  track  enters  or  leaves  a  particular  group,  all 
tracks  in  the  group  are  terminated  and  re-initialized.  The  resulting  set  of  segmented  tracks  is  then  passed 
to  a  third  processing  stage:  an  MFIT  stage  (with  n-scan»0)  that  performs  segment-to-segment  fusion  with 
the  inclusion  of  low-revisit  rate  feature-rich  data.  This  processing  scheme  is  closely  related  to  that 
developed  in  [7],  but  without  the  need  for  aggregate  group  state  computations. 

The  track-segment-fuse  approach  is  preferable  to  the  track-break-fuse  approach  when  a  high  down-stream 
n-scan  is  required  or  when  multi-target  association  ambiguities  persist  for  many  sensor  scans.  The 
modified  fuse-before-track  approaches  (see  table  3.1)  are  of  intermediate  complexity  between  the  others, 
as  track  breaks  are  introduced  for  at  each  time  step:  fewer  breaks  than  in  track-break-fuse,  but  more  than 
in  track-segment-fuse. 
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3.2  Tracking  with  low-resolution  sensors  ( track-adapt-track ) 

During  recent  NURC  at-sea  experimentation,  the  modified  track-before-fuse  architecture  was  severely 
tested  due  to  the  limited  resolution  of  legacy  ship-based  radar  systems  [8].  In  particular,  we  find  that 
vessel  radar  tracks  are  recently  dropped  when  in  close  proximity  to  another.  Subsequent  to  the  at-sea 
experimentation,  we  have  examined  this  problem  further  in  a  simulation-based  setting.  In  particular,  we 
simulate  a  rendezvous-at-sea  whereby  targets  approach,  move  in  close  proximity  for  some  time,  and  then 
diverge.  Contact  data  is  simulated  with  a  nominal  FAR  but  with  a  PD  that  drops  significantly  while  the 
targets  are  close. 

One  approach  to  this  problem  is  to  consider  an  adaptive  track  coasting  scheme  whereby  tracks  in  close 
proximity  are  coasted  for  much  longer  times  in  the  absence  of  a  measurement  update.  This  approach  is 
problematic,  as  tracks  are  easily  seduced  by  false  contacts.  A  more  robust  approach  is  to  allow  track 
terminations  after  short- duration  coasts,  followed  by  adaptive  coasting  in  a  second  stage.  An  illustration 
of  the  effectiveness  of  the  two-stage,  track-adapt-track  approach  is  shown  in  figures  3. 1-3.2. 

There  is  still  a  need  for  further  improvement  to  the  track-adaptive-track  scheme.  Indeed,  note  in  the 
example  that,  while  we  are  able  successfully  to  associate  two  track  segments  on  either  side  of  the  close- 
target  motion,  the  other  two  segments  are  not  successfully  associated.  Indeed,  it  is  necessary  to  augment 
the  second-stage  adaptive  coasting  so  that  the  current  coasting  methodology  based  on  a  nearly-constant 
velocity  target  motion  model  (which  amounts  to  dead-reckoning)  be  replaced  by  a  more  sophisticated 
approach  that  coasts  the  track  in  vicinity  of  existing  active  tracks,  where  these  exist.  This  would  allow  for 
maneuvers  as  part  of  the  track-coasting  process. 

3.3  Tracking  dim  targets  in  clutter  {low-q  track  -  high-q  track) 

Tracking  low  PD  targets  in  clutter  is  a  significant  challenge.  Maximum  likelihood  (ML)  approaches 
effectively  lower  the  target  process  noise  to  zero,  and  limit  the  search  to  a  single  non-manuevering  target. 
Recently,  there  has  been  work  to  extend  ML-based  approaches  to  handle  multiple,  maneuvering  target. 

The  MHT  solution  provides  a  useful  tool  to  address  the  dim-target  challenge.  In  particular,  we  proceed  in 
a  two-stage  approach.  In  the  first  stage,  we  process  the  data  with  near-zero  process  noise.  In  the  second 
stage,  we  allow  for  much  lengthier  track  coasts  (as  previously  done,  this  is  best  performed  in  a 
downstream  stage  with  less  clutter),  as  well  as  with  a  larger  process  noise.  The  idea  is  to  associate  short, 
low-maneuver  segments  into  higher-continuity  tracks.  An  illustration  of  the  potential  of  this  approach  is 
given  in  figures  3. 3 -3. 6. 
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Figure  3.1:  Crossing  target  trajectories  (magenta),  contact  data  (black),  single-stage  adaptive 
tracking  (red),  first-stage  tracking  (blue),  track-adapt-track  ( cyan). 


red=centralized,  blue=first  stage,  cyan=second  stage 


Figure  3.2:  Close-up  of  crossing  target  trajectories  (magenta),  contact  data  (black),  single-stage 
adaptive  tracking  (red),  first-stage  tracking  (blue),  track-adapt-track  (cyan). 
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red=centralized,  blue=first  stage,  cyan=second  stage 


5 


0 


-5 


-10 


Figure  3.3:  The  dim-target  problem.  Target  trajectories  (magenta),  contact  data  (black),  single- 

stage  tracking  (red). 


red=centralized,  blue=first  stage,  cyan=second  stage 


Figure  3.4:  The  dim-target  problem.  Target  trajectories  (magenta),  contact  data  (black),  low- 

process-noise  tracking  (blue). 
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red=centralized,  blue=first  stage,  cyan=second  stage 


Figure  3.5:  The  dim-target  problem.  Target  trajectories  (magenta),  contact  data  (black),  second- 
stage  tracking  (cyan)  in  low-q  track  -  high-q  track  architecture. 


red=centralized,  blue=first  stage,  cyan=second  stage 


Figure  3.6:  The  dim-target  problem:  aggregate  results.  Target  trajectories  (magenta),  contact 
data  (black),  low-process-noise  tracking  (blue),  second-stage  tracking  (cyan)  in  low-q  track - 

high-q  track  architecture. 
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3.4  Track  merging 

Automatic  tracking  often  results  in  track  fragmentation  and  redundant  tracks  on  the  same  target.  This  may 
be  caused  by  sensor  limitations  (missed  detections,  high  false  contact  rate,  unresolved  measurement, 
redundant  measurements)  or  tracker  limitations.  It  is  of  interest  to  develop  schemes  to  improve  track 
continuity. 

Our  approach  to  this  problem  is  to  consider  a  post-tracker,  track-merging  algorithm.  There  are  a  number 
of  reasons  for  choosing  this  two-stage  processing  paradigm  as  opposed  to  seeking  to  develop  an  enhanced, 
single-stage  tracking  solution: 

•  This  is  a  simpler  approach  as  it  does  not  require  changing  the  fundamental  assumption  that  targets 
give  rise  to  at  most  one  measurement  per  scan; 

•  More  aggressive  fusion  (in  this  context,  allowing  for  multiple  target  returns  from  the  same  object) 
is  less  error  prone  in  downstream  processing  in  which  false  returns  are  largely  absent; 

•  The  tracker  is  scan-based,  while  our  post-processing  allows  for  batch  analysis. 

Simplifying  assumptions: 

•  Single-sensor,  single-footprint  formulation;  the  general  case  will  involve  a  multi-sensor,  multi¬ 
footprint  formulation; 

•  Upstream  tracker  involves  hard  data  association  and  sensor  measurement  information  is  available 
at  output  (correspondingly,  terminal  track  coasts  are  removed); 

•  No  prior  distribution  on  number  of  targets. 

We  develop  a  sub-optimal,  greedy  fusion  approach  as  the  optimal  solution  may  be  computationally 
intractable  for  a  large  number  of  tracks.  We  assume  knowledge  of  the  following: 

•  Target  kinematic  model  and  sensor  measurement  model  that  allow  for  the  evaluation  of 
p[xi  |  x‘  1 ),  where  xl  1  is  the  measurement  (and  missed  detection)  sequence  up  to  and  including 
time  fM; 

•  A  sensor  detection  model  for  the  number  of  returns  per  target  per  scan  at  time  /, .  In  a  more 
general  form,  this  model  is  the  convolution  of  Bernoulli  and  Poisson  distributions,  allowing  for  a 
target-originated  contact  with  probability  PD  as  well  as  a  Poisson-distributed  number  of  false 
returns.  The  distribution  may  be  simplified  by  limiting  to  one  the  number  of  extraneous  returns 
(with  probability  Pfa  )  ;  accordingly,  we  may  then  use  the  multinomial  distribution  as  follows: 

pM=^-Pd\^-pFa),  pa§=pDb-pFA)+pFAiy-pD),™a  p,„ (2)  =  pdpfa ■ 

Note  that  the  sensor  model  applies  downstream  of  target  tracking,  and  as  such  accounts  for  tracker 
inaccuracies  that  give  rise  to  redundant  measurements.  Thus,  PFA  reflects  sensor  measurement 
redundancies  as  well  as  false  contacts  that  are  associated  to  target  tracks  through  tracker  processing. 
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3.4.1  Track  data  without  sensor  measurements:  the  case  of  partial  information  and  linear 
dynamics  and  measurements 

Most  tracking  systems  do  not  provide  upstream  sensor  measurement  information.  In  fact,  for  those 
trackers  that  utilize  soft  data  association,  generally  there  are  multiple  sensor  measurements  that  give  rise  to 
each  track  update.  Thus,  it  is  of  interest  to  extend  the  basic  track-merging  approach  to  account  for  these 
cases.  As  an  intermediate  case,  trackers  may  provide  partial  sensor  measurement  information;  namely,  an 
indication  as  to  where  a  given  track  state  constitutes  a  predicted  state  (i.e.  track  coast)  or  updated  state  (i.e. 
measurement  available). 

We  will  focus  on  the  case  of  partial  measurement  information  and  linear  dynamical  and  measurement 
models.  In  this  case,  provided  that  tracker  filter  parameters  are  known  and  that  there  is  exactly  one 
measurement  associated  with  each  track  update,  pseudo-measurements  can  be  derived  that  correspond 
exactly  to  the  upstream  measurements. 

Let  the  known  sequence  of  state  estimates  be  given  by  (a(1  1 1), x(2  |  2),...),  and  let  z  =  (zj,z2,...)  be  the 
measurement  indicator  sequence,  i.e.  zl  =  1  if  there  is  a  measurement  at  tl ,  and  z;  =0  otherwise. 
Assume  that  the  target  dynamical  model  and  sensor  measurement  model  are  known: 


4+, 

=  AkXk+wk,wk  ~N(0,Qk), 

(3.1) 

4+ 1 

=  CkXk+vk,wk  ~N(0,Rk). 

(3.2) 

Assume 

4  —  [f'u  4,/t  ] 

with 

CX  k  >  0  and  C2  k  =  0 . 

Correspondingly,  let 

X(k  |  k)  =  [A,  (k  |  k )  A,  (k  |  &)]  .  Let  V  be  the  covariance  prior  for  the  unmeasured  portion  of  the  state 
vector;  as  with  the  target  and  sensor  models,  we  assume  this  filter  parameter  to  be  known.  Manipulation 
of  the  Kalman  filter  equations  leads  directly  to  the  measurement-reconstruction  equations. 


First,  the  filter  covariance  expressions  can  be  reconstructed  without  filter  state  estimates  as  follows: 


P(l|l): 


c;\rc;\ 

o 


cx 

H.l  ■ 


P(k  +  \\k)  =  AkP(k\k)A'k+Q 


k  9 


4+i  =P{k  + 1 1  k)C'k+l  (< Ck+1P(k  + 1 1  k)C'k+]  +  Rk  Yl 
P{k  + 1 1  k  + 1)  =  (/  -  l[z,+i  =  1  ]Lt+A+1  )P{k  + 1 1  k) . 


(3.3) 

(3.4) 

(3.5) 

(3.6) 


Next,  measurement  reconstruction  is  as  follows;  note  that  equation  (8)  applies  when  zk+l  =  1 . 


4  =4;,' 4(1 1 1),  (3.7) 

zk+l  =  ( p(k  +  l|A-)+4t)T'~1(£  +  l|  k){x  {k  + 1 1  k  + 1)  +  (/  -  p{k  +  1 1  k)c'k  (. p{k  + 1 1  k)  +  Rk ]AkX(k  \  kj). 

(3.8) 
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3.4.2  Track  scores 

Let  track  j  be  given  by  a  sequence  of  associated  measurements  (x1,x2,...)  defined  on  (q,t2,...).  We  define 
xt  —  <f>  when  no  measurement  exists  at  time  tr  Further,  let  s(j)  =  min|/(  |  jc.  ^  tf>\  and 

i 

fU )  =  max{t(-  \x(  ^  (f)\ .  The  track  score  c[j)  is  given  by  the  following,  where  x‘  1  denotes  all 

i 

measurements  up  to  and  including  time  : 

c(j)  =  -  log  Y[  fe  *  $Pm  (O/k  I  x'-' )  +  ][x,  =  <f>]pm  (0)}.  (3.9) 

•=4j) 

In  equation  (3.1)  we  have  p{x  t)  =  g(0,  li] ) ,  where  g  is  the  Gaussian  probability  density  function  (pdf). 
Further,  pm  (■)  is  the  multinomial  distribution  as  noted  previously. 

More  generally,  let  track  j  be  given  by  a  sequence  of  associated  measurements  [xxl,xX2,...,x2l,...,xik,..^) 
where  n,  >  0  measurements  may  exist  at  time  tt .  In  the  case  of  no  measurements  at  time  ,  we  have 
xn  =  (j> .  The  track  score  c(j)  is  given  by: 

c0')  =  -lo§  nfki  />(*/,* \xi~\xux,...xik_x)p(xil\x,l)+  l[x,  =  *>„(<>)}■  (3.10) 

i=s(j)[  k= 2  J 

In  equation  (3.10),  we  have  p(xlx )  =  g(0,  ) .  Simplifying  the  notation  by  suppressing  the  redundant- 

measurement  index  i,  the  kinematic -residual  cost  term  is  given  by: 

?{xk+l  I  X* )  =  f(xk+l  -  Ck+]x{k  + 1 1  k\ Ck+lP(k  + 1 1  k)C'M  +  Rk+l ) .  (3.11) 

Note  that,  in  general,  the  track  score  depends  on  the  ordering  of  multiple  measurements  at  the  same  time. 
We  neglect  this  and  consider  an  arbitrary  ordering  of  such  measurement.  (In  the  case  of  linear 
measurements,  the  ordering  does  not  impact  the  track  score.) 

3.4.3  Greedy  track  merging 

Let  ./V  be  the  number  of  tracks  of  interest,  and  let  each  track  j  have  associated  cost  c{  j) .  We  consider  an 
N-by-N  matrix  C  of  fused-track  costs,  where  the  ijtb  element  c;/  is  given  by  c(l )  -  c(i )-c(j).  In  turn,  track 
/(/,  j)  is  defined  by  the  merged,  time-ordered  sequences  of  measurements  in  tracks  i  and  j.  The  diagonal 
elements  of  C  are  set  to  zero:  Cjj  =  0 ,j  =  1  ,...N .  Each  matrix  element  c(/  represents  the  cost  of  replacing 
tracks  i  and  j  with  the  corresponding  track  /(/,  j) . 

Having  generated  matrix  C,  we  identify  its  smallest  negative  matrix  element  c-.-. .  Correspondingly,  we 

replace  tracks  i  and  j  by  l(i,j).  This  leads  to  a  set  of  N-\  tracks,  for  which  a  new  matrix  C  is 
determined.  The  procedure  is  iterative,  and  terminates  when  no  non-negative  matrix  elements  remain. 

Note  that  the  track  fusion  methodology  defined  here  avoids  the  enumeration  of  the  2N  -1  local  track 
hypotheses  to  be  considered.  Thus,  there  is  no  guarantee  that  the  resulting  set  of  fused  tracks  corresponds 
to  the  optimal  set  of  local  track  hypotheses.  In  principle,  if  2A  —1  were  not  prohibitive,  one  could 
consider  an  LP-relaxation  or  Lagrangian  relaxation  approach  to  determine  the  optimal  set  of  local 
hypotheses  [9-10].  Track-oriented  relaxation  approaches  obviate  the  need  for  considering  global 
hypotheses;  indeed,  given  N  tracks,  there  are  BN  ways  to  partition  these,  where  the  Bell  number  B N 
grows  very  rapidly  as  a  function  of  [11]. 

3.4.4  Examples  with  two  synthetically-generated  tracks 

Let  us  examine  a  simple  example  in  which  a  linear  target  trajectory  gives  rise  to  two  overlapping-in-time 
tracks.  Following  the  methodology  defined  above,  it  is  of  interest  to  investigate  under  what  conditions  the 
tracks  are  merged.  The  exact  scenario  characteristics  are  summarized  in  table  3.2. 


RTO-EN-SET-1 57(2010) 


4-13 


Multi-Stage  Data  Fusion  in  Security  and  Defence 


ORGANIZATION 


Table  3.2:  Scenario  characteristics  for  the  two-track  example. 


Sensor  revisit  time 

lsec 

Sensor  measurement  covariance  matrix 

"1  0" 

0  1 

m2 

Sensor  detection  probability 

ii 

o 

SO 

Sensor  revisit  time 

lsec 

Duration  of  tracks 

60  scans 

Duration  of  track  overlap 

s  scans 

Track  displacement  bias 

d  m 

Tracker  kinematic  process  noise 

0.01  m2s'3 

Tracker  velocity  prior  covariance 

"1  0" 

0  1 

2  -2 

m  s 

Extraneous  measurement  probability 

Pfa 

A  scenario  realization  is  illustrated  in  figures  3. 7-3. 8,  with  d=  lm,  5=3,  and  Pfa  —  0-1  ■ 


Figure  3.7  Two-track  example.  Nominal  target  locations  are  red  and  blue  dots,  sensor 
measurements  are  crosses,  track  segments  are  trajectories,  fused  track  is  black  trajectory. 

Figure  3.9  provides  Monte  Carlo  results.  Each  data  point  is  based  on  20  realizations.  As  expected,  we 
find  that  tracks  that  have  a  temporal  gap  generally  will  not  fuse,  and  likewise  tracks  with  significant 
temporal  overlap  will  not  fuse.  The  highest  fusion  likelihood  is  achieved  when  there  is  neither  a  temporal 
overlap  nor  a  temporal  gap.  As  the  kinematic  discrepancy  (track  displacement  bias  d)  is  reduced  or  the 
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redundancy  parameter  is  increased  (extraneous  measurement  probability  F*fa  )>  fusion  is  more  likely  to 
lead  to  improved  track  scores. 


Figure  3.8:  Close-up  of  track  overlap  in  figure  3.7. 


Figure  3.9:  Baseline  results  (d=  2.  Pfa  —0-1  )  in  red;  variations  leading  to  higher  probability  of 
track-score  improvement  through  fusion:  first  variation  (d=1,  PFA  =  0.1 )  in  blue  and  second 

variation  (d=  2,  PFA  -  0.2)  in  black. 
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It  is  worth  noting  that  in  the  special  case  of  two  point-tracks,  these  will  not  fuse  regardless  of  whether  they 
is  overlap,  a  temporal  gap,  or  neither.  The  best-scoring  fusion  situation  is  where  there  is  neither  overlap 
nor  a  gap.  Nonetheless,  even  in  this  instance,  the  kinematic-filter  score  given  by  equation  (3.10)  is  slightly 
lower  than  in  the  case  of  a  second  track  initiation.  The  result  is  a  reasonable  one:  tracks  should  fuse  only 
when  the  detection  and  kinematic  characteristics  of  the  scenario  are  such  as  to  justify  modification  of  the 
upstream  tracker  output. 

In  general,  out-of-phase  measurements  (i.e.  coasts  on  one  track  when  the  other  has  updates)  and  single- 
track  maneuvers  in  conjunction  with  track  fragmentation  will  increase  the  likelihood  that  track  merging 
will  be  performed.  Such  instances  emerge  in  more  complex,  data-driven  track  realizations  than  those 
examined  in  this  section.  Thus,  more  comprehensive  analysis  of  track  merging  performance  is  required 
with  tracker  executions  based  on  simulated  sensor  data. 

To  conclude,  our  approach  to  track  merging  accounts  for  target  kinematic  characteristics,  sensor  detection 
and  localization  characteristics,  and  the  impact  of  target  tracking  on  redundant  measurements.  Further 
work  includes:  (1)  analysis  of  the  performance  benefits  of  track  merging  with  simulation-based  tracker 
runs;  (2)  treatment  of  the  general  case  of  legacy  track  data  and  a  quantification  of  the  corresponding 
performance  degradation;  and  (3)  development  of  a  scan-based  track-merging  methodology,  for  use  in 
real-time  tracking  applications. 


4  CONCLUSIONS 

Estimation  and  detection  theory  provide  a  theoretical  underpinning  for  a  wide  variety  of  problems  that, 
taken  as  a  whole,  constitute  the  target  tracking  challenge.  These  notes  provide  a  brief  taxonomy  of 
approaches  to  this  challenge,  and  move  on  to  discuss  specific  results  on  the  target  cardinality  problem  and 
in  multi-stage  tracking ;  the  latter  in  particular  is  an  area  of  active  research  in  which  the  author  has  been 
engaged  for  many  years.  Multi-stage  approaches  to  tracking  are  often  simpler,  more  robust,  and  higher 
performing  than  highly  sophisticated  single-stage  processing  algorithms.  We  hope  to  encourage  more 
research  on  fusion  architectures,  as  we  believe  this  topic  constitutes  a  fruitful  area  for  significant  advances 
in  surveillance  capabilities  for  a  wide  variety  of  security  and  defense  applications. 
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